SherbyNet - /page/wikibook/entry.mu

Micron Document

<!DOCTYPE html>

<title>The Alignment Problem</title>

</head>

<span id="openzim-page-title" class="mw-page-title-main"><i>The Alignment Problem</i></span>

</h1>

</header>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div role="note" class="hatnote navigation-not-searchable">This article is about the book. For the alignment problem in artificial intelligence, see <a href="AI_alignment" title="AI alignment">AI alignment</a>.</div>

</p>

/* start https://en.wikipedia.org/ */

.mw-parser-output .infobox-subbox{padding:0;border:none;margin:-3px;width:auto;min-width:100%;font-size:100%;clear:none;float:none;background-color:transparent}.mw-parser-output .infobox-3cols-child{margin:auto}.mw-parser-output .infobox .navbar{font-size:100%}@media screen{html.skin-theme-clientpref-night .mw-parser-output .infobox-full-data:not(.notheme)>div:not(.notheme)[style]{background:#1f1f23!important;color:#f8f9fa}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .infobox-full-data:not(.notheme)>div:not(.notheme)[style]{background:#1f1f23!important;color:#f8f9fa}}@media(min-width:640px){body.skin--responsive .mw-parser-output .infobox-table{display:table!important}body.skin--responsive .mw-parser-output .infobox-table>caption{display:table-caption!important}body.skin--responsive .mw-parser-output .infobox-table>tbody{display:table-row-group}body.skin--responsive .mw-parser-output .infobox-table th,body.skin--responsive .mw-parser-output .infobox-table td{padding-left:inherit;padding-right:inherit}}

/* end https://en.wikipedia.org/ */

</style><style data-mw-deduplicate="TemplateStyles:r1287740527">

/* start https://en.wikipedia.org/ */

.mw-parser-output .ib-book .infobox-title{font-style:italic;padding-bottom:0.2em}.mw-parser-output .ib-book .Z3988{display:none}.mw-parser-output .ib-book .infobox-below{border-top:#aaa 1px solid}

/* end https://en.wikipedia.org/ */

</style><table class="infobox ib-book vcard"><caption class="infobox-title">The Alignment Problem: Machine Learning and Human Values </caption><tbody><tr><td colspan="2" class="infobox-image"><div class="infobox-caption">Hardcover edition</div></td></tr><tr><th scope="row" class="infobox-label">Author</th><td class="infobox-data"><a href="Brian_Christian" title="Brian Christian">Brian Christian</a></td></tr><tr><th scope="row" class="infobox-label">Language</th><td class="infobox-data">English</td></tr><tr><th scope="row" class="infobox-label">Subject</th><td class="infobox-data"><a href="AI_alignment" title="AI alignment">AI alignment</a></td></tr><tr><th scope="row" class="infobox-label">Publisher</th><td class="infobox-data"><a href="W._W._Norton_%26_Company" title="W. W. Norton & Company">W. W. Norton & Company</a><sup id="cite_ref-1" class="reference"><a href="#cite_note-1"><span class="cite-bracket">[</span>1<span class="cite-bracket">]</span></a></sup></td></tr><tr><th scope="row" class="infobox-label"><div style="display: inline-block; line-height: 1.2em; padding: .1em 0;">Publication date</div></th><td class="infobox-data">October 6, 2020</td></tr><tr><th scope="row" class="infobox-label">Publication place</th><td class="infobox-data">United States</td></tr><tr><th scope="row" class="infobox-label">Media type</th><td class="infobox-data">Print, e-book, audiobook</td></tr><tr><th scope="row" class="infobox-label">Pages</th><td class="infobox-data">496</td></tr><tr><th scope="row" class="infobox-label"><a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a></th><td class="infobox-data"><style data-mw-deduplicate="TemplateStyles:r1238218222">

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><bdi>0393635821</bdi></td></tr><tr><th scope="row" class="infobox-label"><a href="OCLC_(identifier)" class="mw-redirect" title="OCLC (identifier)"><abbr title="Online Computer Library Center number">OCLC</abbr></a></th><td class="infobox-data"><a rel="nofollow" class="external text" href="https://www.worldcat.org/oclc/1137850003">1137850003</a></td></tr><tr><th scope="row" class="infobox-label">Website</th><td class="infobox-data"><span class="url"><a rel="nofollow" class="external text" href="https://brianchristian.org/the-alignment-problem/">brianchristian.org/the-alignment-problem/</a></span></td></tr></tbody></table>

<p><i><b>The Alignment Problem: Machine Learning and Human Values</b></i> is a 2020 non-fiction book by the American writer <a href="Brian_Christian" title="Brian Christian">Brian Christian</a>. It is based on numerous interviews with experts trying to build <a href="Artificial_intelligence" title="Artificial intelligence">artificial intelligence</a> systems, particularly <a href="Machine_learning" title="Machine learning">machine learning</a> systems, that are <a href="AI_alignment" title="AI alignment">aligned</a> with human values.

</p>

<div class="mw-heading mw-heading2"><h2 id="Summary">Summary</h2></div>

<p>The book is divided into three sections: Prophecy, Agency, and Normativity. Each section covers researchers and engineers working on different challenges in the alignment of <a href="Artificial_intelligence" title="Artificial intelligence">artificial intelligence</a> with human values.

</p>

<div class="mw-heading mw-heading3"><h3 id="Prophecy">Prophecy</h3></div>

<p>In the first section, Christian interweaves discussions of the history of artificial intelligence research, particularly the <a href="Machine_learning" title="Machine learning">machine learning</a> approach of <a href="Artificial_neural_networks" class="mw-redirect" title="Artificial neural networks">artificial neural networks</a> such as the <a href="Perceptron" title="Perceptron">Perceptron</a> and <a href="AlexNet" title="AlexNet">AlexNet</a>, with examples of how AI systems can have unintended behavior. He tells the story of <a href="Julia_Angwin" title="Julia Angwin">Julia Angwin</a>, a journalist whose <a href="ProPublica" title="ProPublica">ProPublica</a> investigation of the <a href="COMPAS_(software)" title="COMPAS (software)">COMPAS</a> algorithm, a tool for predicting <a href="Recidivism" title="Recidivism">recidivism</a> among criminal defendants, led to widespread criticism of its accuracy and bias towards certain demographics. One of AI's main alignment challenges is its <a href="Black_box" title="Black box">black box</a> nature (inputs and outputs are identifiable but the transformation process in between is undetermined). The lack of transparency makes it difficult to know where the system is going right and where it is going wrong.

</p>

<div class="mw-heading mw-heading3"><h3 id="Agency">Agency</h3></div>

<p>In the second section, Christian similarly interweaves the history of the <a href="Psychological" class="mw-redirect" title="Psychological">psychological</a> study of reward, such as <a href="Behaviorism" title="Behaviorism">behaviorism</a> and <a href="Dopamine" title="Dopamine">dopamine</a>, with the computer science of <a href="Reinforcement_learning" title="Reinforcement learning">reinforcement learning</a>, in which AI systems need to develop policy ("what to do") in the face of a value function ("what rewards or punishment to expect"). He calls the <a href="DeepMind" class="mw-redirect" title="DeepMind">DeepMind</a> <a href="AlphaGo" title="AlphaGo">AlphaGo</a> and <a href="AlphaZero" title="AlphaZero">AlphaZero</a> systems "perhaps the single most impressive achievement in automated curriculum design." He also highlights the importance of curiosity, in which reinforcement learners are intrinsically motivated to explore their environment, rather than exclusively seeking the external reward.

</p>

<div class="mw-heading mw-heading3"><h3 id="Normativity">Normativity</h3></div>

<p>The third section covers training AI through the imitation of human or machine behavior, as well as philosophical debates such as between <a href="Possibilism_(philosophy)" class="mw-redirect" title="Possibilism (philosophy)">possibilism</a> and <a href="Actualism" title="Actualism">actualism</a> that imply different ideal behavior for AI systems. Of particular importance is <a href="Inverse_reinforcement_learning" class="mw-redirect" title="Inverse reinforcement learning">inverse reinforcement learning</a>, a broad approach for machines to learn the objective function of a human or another agent. Christian discusses the <a href="Normative" class="mw-redirect" title="Normative">normative</a> challenges associated with <a href="Effective_altruism" title="Effective altruism">effective altruism</a> and <a href="Existential_risk" class="mw-redirect" title="Existential risk">existential risk</a>, including the work of philosophers <a href="Toby_Ord" title="Toby Ord">Toby Ord</a> and <a href="William_MacAskill" title="William MacAskill">William MacAskill</a> who are trying to devise human and machine strategies for navigating the alignment problem as effectively as possible.

</p>

<div class="mw-heading mw-heading2"><h2 id="Reception">Reception</h2></div>

<p>The book received positive reviews from critics. <i><a href="The_Wall_Street_Journal" title="The Wall Street Journal">The Wall Street Journal</a></i>'s David A. Shaywitz emphasized the frequent problems when applying algorithms to real-world problems, describing the book as "a nuanced and captivating exploration of this white-hot topic."<sup id="cite_ref-shaywitz_2-0" class="reference"><a href="#cite_note-shaywitz-2"><span class="cite-bracket">[</span>2<span class="cite-bracket">]</span></a></sup> <i><a href="Publishers_Weekly" title="Publishers Weekly">Publishers Weekly</a></i> praised the book for its writing and extensive research.<sup id="cite_ref-3" class="reference"><a href="#cite_note-3"><span class="cite-bracket">[</span>3<span class="cite-bracket">]</span></a></sup>

</p><p><i><a href="Kirkus_Reviews" title="Kirkus Reviews">Kirkus Reviews</a></i> gave the book a positive review, calling it "technically rich but accessible", and "an intriguing exploration of AI."<sup id="cite_ref-4" class="reference"><a href="#cite_note-4"><span class="cite-bracket">[</span>4<span class="cite-bracket">]</span></a></sup> Writing for <i><a href="Nature_(journal)" title="Nature (journal)">Nature</a></i>, Virginia Dignum gave the book a positive review, favorably comparing it to <a href="Kate_Crawford" title="Kate Crawford">Kate Crawford</a>'s <i><a href="Atlas_of_AI" title="Atlas of AI">Atlas of AI</a></i>.<sup id="cite_ref-5" class="reference"><a href="#cite_note-5"><span class="cite-bracket">[</span>5<span class="cite-bracket">]</span></a></sup>

</p><p>In 2021, journalist <a href="Ezra_Klein" title="Ezra Klein">Ezra Klein</a> had Christian on his podcast, <i>The Ezra Klein Show,</i> writing in <i><a href="The_New_York_Times" title="The New York Times">The New York Times</a></i>, "<i>The Alignment Problem</i> is the best book on the key technical and moral questions of A.I. that I’ve read."<sup id="cite_ref-klein_6-0" class="reference"><a href="#cite_note-klein-6"><span class="cite-bracket">[</span>6<span class="cite-bracket">]</span></a></sup> Later that year, the book was listed in a <i><a href="Fast_Company" title="Fast Company">Fast Company</a></i> feature, "5 books that inspired Microsoft CEO <a href="Satya_Nadella" title="Satya Nadella">Satya Nadella</a> this year".<sup id="cite_ref-nadella_7-0" class="reference"><a href="#cite_note-nadella-7"><span class="cite-bracket">[</span>7<span class="cite-bracket">]</span></a></sup>

</p><p>In 2022, the book won the <a href="Eric_and_Wendy_Schmidt_Award_for_Excellence_in_Science_Communication" class="mw-redirect" title="Eric and Wendy Schmidt Award for Excellence in Science Communication">Eric and Wendy Schmidt Award for Excellence in Science Communication</a>, given by <a href="The_National_Academies_of_Sciences%2C_Engineering%2C_and_Medicine" class="mw-redirect" title="The National Academies of Sciences, Engineering, and Medicine">The National Academies of Sciences, Engineering, and Medicine</a> in partnership with <a href="Schmidt_Futures" title="Schmidt Futures">Schmidt Futures</a>.<sup id="cite_ref-8" class="reference"><a href="#cite_note-8"><span class="cite-bracket">[</span>8<span class="cite-bracket">]</span></a></sup>

</p><p>In 2024, <i><a href="The_New_York_Times" title="The New York Times">The New York Times</a></i> placed <i>The Alignment Problem</i> first in its list of the "5 Best Books About Artificial Intelligence," saying: "If you're going to read one book on artificial intelligence, this is the one."<sup id="cite_ref-nyt5best_9-0" class="reference"><a href="#cite_note-nyt5best-9"><span class="cite-bracket">[</span>9<span class="cite-bracket">]</span></a></sup>

</p>

<ul><li><a href="Effective_altruism" title="Effective altruism">Effective altruism</a></li>

<li><a href="Global_catastrophic_risk" title="Global catastrophic risk">Global catastrophic risk</a></li>

<li><i><a href="Human_Compatible" title="Human Compatible">Human Compatible: Artificial Intelligence and the Problem of Control</a></i></li>

<li><i><a href="Superintelligence%3A_Paths%2C_Dangers%2C_Strategies" title="Superintelligence: Paths, Dangers, Strategies">Superintelligence: Paths, Dangers, Strategies</a></i></li></ul>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist reflist-columns references-column-width" style="column-width: 30em;">

<li id="cite_note-1"><span class="mw-cite-backlink"><b><a href="#cite_ref-1">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://wwnorton.com/books/9780393635829">"The Alignment Problem"</a>. <i><a href="W._W._Norton_%26_Company" title="W. W. Norton & Company">W. W. Norton & Company</a></i>.</cite></span>

</li>

<li id="cite_note-shaywitz-2"><span class="mw-cite-backlink"><b><a href="#cite_ref-shaywitz_2-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFShaywitz2020" class="citation news cs1">Shaywitz, David (25 October 2020). <a rel="nofollow" class="external text" href="https://www.wsj.com/articles/the-alignment-problem-review-when-machines-miss-the-point-11603659140">"'The Alignment Problem' Review: When Machines Miss the Point"</a>. <a href="The_Wall_Street_Journal" title="The Wall Street Journal">The Wall Street Journal</a><span class="reference-accessdate">. Retrieved <span class="nowrap">5 December</span> 2021</span>.</cite></span>

</li>

<li id="cite_note-3"><span class="mw-cite-backlink"><b><a href="#cite_ref-3">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.publishersweekly.com/978-0-393-63582-9">"Nonfiction Book Review: The Alignment Problem: Machine Learning and Human Values by Brian Christian. Norton, $27.95 (356p) ISBN 978-0-393-63582-9"</a>. <i>PublishersWeekly.com</i><span class="reference-accessdate">. Retrieved <span class="nowrap">20 January</span> 2022</span>.</cite></span>

</li>

<li id="cite_note-4"><span class="mw-cite-backlink"><b><a href="#cite_ref-4">^</a></b></span> <span class="reference-text"><cite class="citation book cs1"><a rel="nofollow" class="external text" href="https://www.kirkusreviews.com/book-reviews/brian-christian/alignment-problem/"><i>THE ALIGNMENT PROBLEM | Kirkus Reviews</i></a>.</cite></span>

</li>

<li id="cite_note-5"><span class="mw-cite-backlink"><b><a href="#cite_ref-5">^</a></b></span> <span class="reference-text"><cite id="CITEREFDignum2021" class="citation journal cs1">Dignum, Virginia (26 May 2021). <a rel="nofollow" class="external text" href="https://doi.org/10.1038%2Fd41586-021-01397-x">"AI — the people and places that make, use and manage it"</a>. <i>Nature</i>. <b>593</b> (7860): <span class="nowrap">499–</span>500. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2021Natur.593..499D">2021Natur.593..499D</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.1038%2Fd41586-021-01397-x">10.1038/d41586-021-01397-x</a></span>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:235216649">235216649</a>.</cite></span>

</li>

<li id="cite_note-klein-6"><span class="mw-cite-backlink"><b><a href="#cite_ref-klein_6-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFKlein2021" class="citation news cs1">Klein, Ezra (4 June 2021). <a rel="nofollow" class="external text" href="https://www.nytimes.com/2021/06/04/opinion/ezra-klein-podcast-brian-christian.html">"If 'All Models Are Wrong,' Why Do We Give Them So Much Power?"</a>. <i><a href="The_New_York_Times" title="The New York Times">The New York Times</a></i><span class="reference-accessdate">. Retrieved <span class="nowrap">5 December</span> 2021</span>.</cite></span>

</li>

<li id="cite_note-nadella-7"><span class="mw-cite-backlink"><b><a href="#cite_ref-nadella_7-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFNadella2021" class="citation news cs1">Nadella, Satya (15 November 2021). <a rel="nofollow" class="external text" href="https://www.fastcompany.com/90696770/microsoft-satya-nadella-book-recommendations">"5 books that inspired Microsoft CEO Satya Nadella this year"</a>. <a href="Fast_Company" title="Fast Company">Fast Company</a><span class="reference-accessdate">. Retrieved <span class="nowrap">5 December</span> 2021</span>.</cite></span>

</li>

<li id="cite_note-8"><span class="mw-cite-backlink"><b><a href="#cite_ref-8">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.nationalacademies.org/awards/excellence-in-communication/winners">"Winners - Eric and Wendy Schmidt Awards for Excellence in Science Communication - National Academies"</a>. <i><a href="National_Academies_of_Sciences%2C_Engineering%2C_and_Medicine" title="National Academies of Sciences, Engineering, and Medicine">National Academies</a></i>. 12 October 2022<span class="reference-accessdate">. Retrieved <span class="nowrap">21 October</span> 2022</span>.</cite></span>

</li>

<li id="cite_note-nyt5best-9"><span class="mw-cite-backlink"><b><a href="#cite_ref-nyt5best_9-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFMarche2024" class="citation news cs1">Marche, Stephen (31 January 2024). <a rel="nofollow" class="external text" href="https://www.nytimes.com/2024/01/31/books/review/artificial-intelligence-best-books.html">"5 Best Books About Artificial Intelligence"</a>. <i><a href="New_York_Times" class="mw-redirect" title="New York Times">New York Times</a></i><span class="reference-accessdate">. Retrieved <span class="nowrap">6 February</span> 2024</span>.</cite></span>

</li>

</ol></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hlist dl,.mw-parser-output .hlist ol,.mw-parser-output .hlist ul{margin:0;padding:0}.mw-parser-output .hlist dd,.mw-parser-output .hlist dt,.mw-parser-output .hlist li{margin:0;display:inline}.mw-parser-output .hlist.inline,.mw-parser-output .hlist.inline dl,.mw-parser-output .hlist.inline ol,.mw-parser-output .hlist.inline ul,.mw-parser-output .hlist dl dl,.mw-parser-output .hlist dl ol,.mw-parser-output .hlist dl ul,.mw-parser-output .hlist ol dl,.mw-parser-output .hlist ol ol,.mw-parser-output .hlist ol ul,.mw-parser-output .hlist ul dl,.mw-parser-output .hlist ul ol,.mw-parser-output .hlist ul ul{display:inline}.mw-parser-output .hlist .mw-empty-li{display:none}.mw-parser-output .hlist dt::after{content:": "}.mw-parser-output .hlist dd::after,.mw-parser-output .hlist li::after{content:" · ";font-weight:bold}.mw-parser-output .hlist dd:last-child::after,.mw-parser-output .hlist dt:last-child::after,.mw-parser-output .hlist li:last-child::after{content:none}.mw-parser-output .hlist dd dd:first-child::before,.mw-parser-output .hlist dd dt:first-child::before,.mw-parser-output .hlist dd li:first-child::before,.mw-parser-output .hlist dt dd:first-child::before,.mw-parser-output .hlist dt dt:first-child::before,.mw-parser-output .hlist dt li:first-child::before,.mw-parser-output .hlist li dd:first-child::before,.mw-parser-output .hlist li dt:first-child::before,.mw-parser-output .hlist li li:first-child::before{content:" (";font-weight:normal}.mw-parser-output .hlist dd dd:last-child::after,.mw-parser-output .hlist dd dt:last-child::after,.mw-parser-output .hlist dd li:last-child::after,.mw-parser-output .hlist dt dd:last-child::after,.mw-parser-output .hlist dt dt:last-child::after,.mw-parser-output .hlist dt li:last-child::after,.mw-parser-output .hlist li dd:last-child::after,.mw-parser-output .hlist li dt:last-child::after,.mw-parser-output .hlist li li:last-child::after{content:")";font-weight:normal}.mw-parser-output .hlist ol{counter-reset:listitem}.mw-parser-output .hlist ol>li{counter-increment:listitem}.mw-parser-output .hlist ol>li::before{content:" "counter(listitem)"\a0 "}.mw-parser-output .hlist dd ol>li:first-child::before,.mw-parser-output .hlist dt ol>li:first-child::before,.mw-parser-output .hlist li ol>li:first-child::before{content:" ("counter(listitem)"\a0 "}

/* end https://en.wikipedia.org/ */

</style><style data-mw-deduplicate="TemplateStyles:r1236075235">

/* start https://en.wikipedia.org/ */

.mw-parser-output .navbox{box-sizing:border-box;border:1px solid #a2a9b1;width:100%;clear:both;font-size:88%;text-align:center;padding:1px;margin:1em auto 0}.mw-parser-output .navbox .navbox{margin-top:0}.mw-parser-output .navbox+.navbox,.mw-parser-output .navbox+.navbox-styles+.navbox{margin-top:-1px}.mw-parser-output .navbox-inner,.mw-parser-output .navbox-subgroup{width:100%}.mw-parser-output .navbox-group,.mw-parser-output .navbox-title,.mw-parser-output .navbox-abovebelow{padding:0.25em 1em;line-height:1.5em;text-align:center}.mw-parser-output .navbox-group{white-space:nowrap;text-align:right}.mw-parser-output .navbox,.mw-parser-output .navbox-subgroup{background-color:#fdfdfd}.mw-parser-output .navbox-list{line-height:1.5em;border-color:#fdfdfd}.mw-parser-output .navbox-list-with-group{text-align:left;border-left-width:2px;border-left-style:solid}.mw-parser-output tr+tr>.navbox-abovebelow,.mw-parser-output tr+tr>.navbox-group,.mw-parser-output tr+tr>.navbox-image,.mw-parser-output tr+tr>.navbox-list{border-top:2px solid #fdfdfd}.mw-parser-output .navbox-title{background-color:#ccf}.mw-parser-output .navbox-abovebelow,.mw-parser-output .navbox-group,.mw-parser-output .navbox-subgroup .navbox-title{background-color:#ddf}.mw-parser-output .navbox-subgroup .navbox-group,.mw-parser-output .navbox-subgroup .navbox-abovebelow{background-color:#e6e6ff}.mw-parser-output .navbox-even{background-color:#f7f7f7}.mw-parser-output .navbox-odd{background-color:transparent}.mw-parser-output .navbox .hlist td dl,.mw-parser-output .navbox .hlist td ol,.mw-parser-output .navbox .hlist td ul,.mw-parser-output .navbox td.hlist dl,.mw-parser-output .navbox td.hlist ol,.mw-parser-output .navbox td.hlist ul{padding:0.125em 0}.mw-parser-output .navbox .navbar{display:block;font-size:100%}.mw-parser-output .navbox-title .navbar{float:left;text-align:left;margin-right:0.5em}body.skin--responsive .mw-parser-output .navbox-image img{max-width:none!important}@media print{body.ns-0 .mw-parser-output .navbox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style></div><div role="navigation" class="navbox" aria-labelledby="Effective_altruism81" style="padding:3px"><table class="nowraplinks mw-collapsible autocollapse navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><style data-mw-deduplicate="TemplateStyles:r1239400231">

/* start https://en.wikipedia.org/ */

.mw-parser-output .navbar{display:inline;font-size:88%;font-weight:normal}.mw-parser-output .navbar-collapse{float:left;text-align:left}.mw-parser-output .navbar-boxtext{word-spacing:0}.mw-parser-output .navbar ul{display:inline-block;white-space:nowrap;line-height:inherit}.mw-parser-output .navbar-brackets::before{margin-right:-0.125em;content:"[ "}.mw-parser-output .navbar-brackets::after{margin-left:-0.125em;content:" ]"}.mw-parser-output .navbar li{word-spacing:-0.125em}.mw-parser-output .navbar a>span,.mw-parser-output .navbar a>abbr{text-decoration:inherit}.mw-parser-output .navbar-mini abbr{font-variant:small-caps;border-bottom:none;text-decoration:none;cursor:inherit}.mw-parser-output .navbar-ct-full{font-size:114%;margin:0 7em}.mw-parser-output .navbar-ct-mini{font-size:114%;margin:0 4em}html.skin-theme-clientpref-night .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}@media(prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}}@media print{.mw-parser-output .navbar{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div id="Effective_altruism81" style="font-size:114%;margin:0 4em"><a href="Effective_altruism" title="Effective altruism">Effective altruism</a></div></th></tr><tr><th scope="row" class="navbox-group" style="width:1%">Concepts</th><td class="navbox-list-with-group navbox-list navbox-odd hlist" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Aid_effectiveness" title="Aid effectiveness">Aid effectiveness</a></li>

<li><a href="Charity_assessment" title="Charity assessment">Charity assessment</a></li>

<li><a href="Demandingness_objection" title="Demandingness objection">Demandingness objection</a></li>

<li><a href="Disability-adjusted_life_year" title="Disability-adjusted life year">Disability-adjusted life year</a></li>

<li><a href="Disease_burden" title="Disease burden">Disease burden</a></li>

<li><a href="Distributional_cost-effectiveness_analysis" title="Distributional cost-effectiveness analysis">Distributional cost-effectiveness analysis</a></li>

<li><a href="Earning_to_give" title="Earning to give">Earning to give</a></li>

<li><a href="Equal_consideration_of_interests" title="Equal consideration of interests">Equal consideration of interests</a></li>

<li><a href="Incremental_cost-effectiveness_ratio" title="Incremental cost-effectiveness ratio">Incremental cost-effectiveness ratio</a></li>

<li><a href="Longtermism" title="Longtermism">Longtermism</a></li>

<li><a href="Marginal_utility" title="Marginal utility">Marginal utility</a></li>

<li><a href="Moral_circle_expansion" title="Moral circle expansion">Moral circle expansion</a></li>

<li><a href="Psychological_barriers_to_effective_altruism" title="Psychological barriers to effective altruism">Psychological barriers to effective altruism</a></li>

<li><a href="Quality-adjusted_life_year" title="Quality-adjusted life year">Quality-adjusted life year</a></li>

<li><a href="Utilitarianism" title="Utilitarianism">Utilitarianism</a></li>

<li><a href="Venture_philanthropy" title="Venture philanthropy">Venture philanthropy</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Key figures</th><td class="navbox-list-with-group navbox-list navbox-even hlist" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Sam_Bankman-Fried" title="Sam Bankman-Fried">Sam Bankman-Fried</a></li>

<li><a href="Liv_Boeree" title="Liv Boeree">Liv Boeree</a></li>

<li><a href="Nick_Bostrom" title="Nick Bostrom">Nick Bostrom</a></li>

<li><a href="Hilary_Greaves" title="Hilary Greaves">Hilary Greaves</a></li>

<li><a href="Holden_Karnofsky" title="Holden Karnofsky">Holden Karnofsky</a></li>

<li><a href="William_MacAskill" title="William MacAskill">William MacAskill</a></li>

<li><a href="Dustin_Moskovitz" title="Dustin Moskovitz">Dustin Moskovitz</a></li>

<li><a href="Yew-Kwang_Ng" title="Yew-Kwang Ng">Yew-Kwang Ng</a></li>

<li><a href="Derek_Parfit" title="Derek Parfit">Derek Parfit</a></li>

<li><a href="Kelsey_Piper" title="Kelsey Piper">Kelsey Piper</a></li>

<li><a href="Peter_Singer" title="Peter Singer">Peter Singer</a></li>

<li><a href="Brian_Tomasik" title="Brian Tomasik">Brian Tomasik</a></li>

<li><a href="Eliezer_Yudkowsky" title="Eliezer Yudkowsky">Eliezer Yudkowsky</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Organizations</th><td class="navbox-list-with-group navbox-list navbox-odd hlist" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="80%2C000_Hours" title="80,000 Hours">80,000 Hours</a></li>

<li><a href="Against_Malaria_Foundation" title="Against Malaria Foundation">Against Malaria Foundation</a></li>

<li><a href="Animal_Charity_Evaluators" title="Animal Charity Evaluators">Animal Charity Evaluators</a></li>

<li><a href="Animal_Ethics_(organization)" title="Animal Ethics (organization)">Animal Ethics</a></li>

<li><a href="Centre_for_Effective_Altruism" title="Centre for Effective Altruism">Centre for Effective Altruism</a></li>

<li><a href="Centre_for_Enabling_EA_Learning_%26_Research" title="Centre for Enabling EA Learning & Research">Centre for Enabling EA Learning & Research</a></li>

<li><a href="Center_for_High_Impact_Philanthropy" title="Center for High Impact Philanthropy">Center for High Impact Philanthropy</a></li>

<li><a href="Centre_for_the_Study_of_Existential_Risk" title="Centre for the Study of Existential Risk">Centre for the Study of Existential Risk</a></li>

<li><a href="Development_Media_International" title="Development Media International">Development Media International</a></li>

<li><a href="Evidence_Action" title="Evidence Action">Evidence Action</a></li>

<li><a href="Faunalytics" title="Faunalytics">Faunalytics</a></li>

<li><a href="Fistula_Foundation" title="Fistula Foundation">Fistula Foundation</a></li>

<li><a href="Future_of_Humanity_Institute" title="Future of Humanity Institute">Future of Humanity Institute</a></li>

<li><a href="Future_of_Life_Institute" title="Future of Life Institute">Future of Life Institute</a></li>

<li><a href="Founders_Pledge" title="Founders Pledge">Founders Pledge</a></li>

<li><a href="GiveDirectly" title="GiveDirectly">GiveDirectly</a></li>

<li><a href="GiveWell" title="GiveWell">GiveWell</a></li>

<li><a href="Giving_Multiplier" title="Giving Multiplier">Giving Multiplier</a></li>

<li><a href="Giving_What_We_Can" title="Giving What We Can">Giving What We Can</a></li>

<li><a href="The_Good_Food_Institute" title="The Good Food Institute">The Good Food Institute</a></li>

<li><a href="Good_Ventures" title="Good Ventures">Good Ventures</a></li>

<li><a href="The_Humane_League" title="The Humane League">The Humane League</a></li>

<li><a href="Mercy_for_Animals" title="Mercy for Animals">Mercy for Animals</a></li>

<li><a href="Machine_Intelligence_Research_Institute" title="Machine Intelligence Research Institute">Machine Intelligence Research Institute</a></li>

<li><a href="Malaria_Consortium" title="Malaria Consortium">Malaria Consortium</a></li>

<li><a href="Open_Philanthropy" title="Open Philanthropy">Open Philanthropy</a></li>

<li><a href="Raising_for_Effective_Giving" title="Raising for Effective Giving">Raising for Effective Giving</a></li>

<li><a href="Sentience_Institute" title="Sentience Institute">Sentience Institute</a></li>

<li><a href="Unlimit_Health" title="Unlimit Health">Unlimit Health</a></li>

<li><a href="Wild_Animal_Initiative" title="Wild Animal Initiative">Wild Animal Initiative</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Focus areas</th><td class="navbox-list-with-group navbox-list navbox-even hlist" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Biotechnology_risk" title="Biotechnology risk">Biotechnology risk</a></li>

<li><a href="Climate_change" title="Climate change">Climate change</a></li>

<li><a href="Cultured_meat" title="Cultured meat">Cultured meat</a></li>

<li><a href="Economic_stability" title="Economic stability">Economic stability</a></li>

<li><a href="Existential_risk_from_artificial_general_intelligence" class="mw-redirect" title="Existential risk from artificial general intelligence">Existential risk from artificial general intelligence</a></li>

<li><a href="Global_catastrophic_risk" title="Global catastrophic risk">Global catastrophic risk</a></li>

<li><a href="Global_health" title="Global health">Global health</a></li>

<li><a href="Global_poverty" class="mw-redirect" title="Global poverty">Global poverty</a></li>

<li><a href="Intensive_animal_farming" title="Intensive animal farming">Intensive animal farming</a></li>

<li><a href="Land_use" title="Land use">Land use reform</a></li>

<li><a href="Life_extension" title="Life extension">Life extension</a></li>

<li><a href="Malaria_prevention" class="mw-redirect" title="Malaria prevention">Malaria prevention</a></li>

<li><a href="Mass_deworming" title="Mass deworming">Mass deworming</a></li>

<li><a href="Neglected_tropical_diseases" title="Neglected tropical diseases">Neglected tropical diseases</a></li>

<li><a href="Risk_of_astronomical_suffering" title="Risk of astronomical suffering">Risk of astronomical suffering</a></li>

<li><a href="Wild_animal_suffering" title="Wild animal suffering">Wild animal suffering</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Literature</th><td class="navbox-list-with-group navbox-list navbox-odd hlist" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><i><a href="Doing_Good_Better" title="Doing Good Better">Doing Good Better</a></i></li>

<li><i><a href="The_End_of_Animal_Farming" title="The End of Animal Farming">The End of Animal Farming</a></i></li>

<li><i><a href="Famine%2C_Affluence%2C_and_Morality" title="Famine, Affluence, and Morality">Famine, Affluence, and Morality</a></i></li>

<li><i><a href="Living_High_and_Letting_Die" title="Living High and Letting Die">Living High and Letting Die</a></i></li>

<li><i><a href="Practical_Ethics" title="Practical Ethics">Practical Ethics</a></i></li>

<li><i><a href="The_Precipice%3A_Existential_Risk_and_the_Future_of_Humanity" title="The Precipice: Existential Risk and the Future of Humanity">The Precipice</a></i></li>

<li><i><a href="Superintelligence%3A_Paths%2C_Dangers%2C_Strategies" title="Superintelligence: Paths, Dangers, Strategies">Superintelligence: Paths, Dangers, Strategies</a></i></li>

<li><i><a href="What_We_Owe_the_Future" title="What We Owe the Future">What We Owe the Future</a></i></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Events</th><td class="navbox-list-with-group navbox-list navbox-even hlist" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Effective_Altruism_Global" title="Effective Altruism Global">Effective Altruism Global</a></li></ul>

</div></td></tr></tbody></table></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-07-20" href="https://en.wikipedia.org/wiki/?title=The_Alignment_Problem&oldid=1301533157">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>